Randomized controlled trial

Flowchart of four phases (enrollment, intervention allocation, follow-up, and data analysis) of a parallel randomized trial of two groups, modified from the CONSORT (Consolidated Standards of Reporting Trials) 2010 Statement[1]

A randomized controlled trial (RCT) is a type of scientific experiment most commonly used in testing the efficacy or effectiveness of healthcare services (such as medicine or nursing) or health technologies (such as pharmaceuticals, medical devices or surgery). RCTs involve the random allocation of different interventions (treatments or conditions) to subjects. The most important advantage of proper randomization is that "it eliminates selection bias, balancing both known and unknown prognostic factors, in the assignment of treatments."[2]

The terms "RCT" and randomized trial are often used synonymously, but some authors distinguish between "RCTs" which compare treatment groups with control groups not receiving treatment (as in a placebo-controlled study), and "randomized trials" which can compare multiple treatment groups with each other.[3] RCTs are sometimes known as randomized control trials.[4] RCTs are also called randomized clinical trials or randomized controlled clinical trials when they concern clinical research[5][6][7]; however, RCTs are also employed in other research areas such as criminology, education, and international development.

Contents

History

It is claimed that the first published RCT was a 1948 paper entitled "Streptomycin treatment of pulmonary tuberculosis. A Medical Research Council investigation."[8][9][10] One of the authors of that paper was Austin Bradford Hill, who is credited as having conceived the modern RCT.[11] However, a recent paper which looked at early RCT use in social and educational intervention studies rather than medical studies found reports of RCT being used as early as 1928.[12]

By the late 20th century, RCTs had become the "gold standard" for "rational therapeutics" in medicine.[13] As of 2004, more than 150,000 RCTs were in the Cochrane Library.[11] To improve the reporting of RCTs in the medical literature, an international group of scientists and editors published Consolidated Standards of Reporting Trials (CONSORT) Statements in 1996, 2001, and 2010 which have become widely accepted.[1][2]

Ethics

Although the principle of clinical equipoise ("genuine uncertainty within the expert medical community... about the preferred treatment") common to clinical trials[14] has been applied to RCTs, the ethics of RCTs have special considerations. For one, it has been argued that equipoise itself is insufficient to justify RCTs.[15] For another, "collective equipoise" can conflict with a lack of personal equipoise (e.g., a personal belief that an intervention is effective).[16] Finally, Zelen's design, which has been used for some RCTs, randomizes subjects before they provide informed consent, which may be ethical for RCTs of screening and selected therapies, but is likely unethical "for most therapeutic trials."[17][18]

Classifications of RCTs

By study design

One way to classify RCTs is by study design. From most to least common in the medical literature, the major categories of RCT study designs are[19]:

An analysis of the 616 RCTs indexed in PubMed during December 2006 found that 78% were parallel-group trials, 16% were crossover, 2% were split-body, 2% were cluster, and 2% were factorial.[19]

By outcome of interest (efficacy vs. effectiveness)

RCTs can be classified as "explanatory" or "pragmatic."[20] Explanatory RCTs test efficacy in a research setting with highly selected participants and under highly controlled conditions.[20] In contrast, pragmatic RCTs test effectiveness in everyday practice with relatively unselected participants and under flexible conditions; in this way, pragmatic RCTs can "inform decisions about practice."[20]

By hypothesis (superiority vs. noninferiority vs. equivalence)

Another classification of RCTs categorizes them as "superiority trials," "noninferiority trials," and "equivalence trials," which differ in methodology and reporting.[21] Most RCTs are superiority trials, in which one intervention is hypothesized to be superior to another in a statistically significant way.[21] Some RCTs are noninferiority trials "to determine whether a new treatment is no worse than a reference treatment."[21] Other RCTs are equivalence trials in which the hypothesis is that two interventions are indistinguishable from each other.[21]

Randomization

The advantages of proper randomization in RCTs include[22]:

There are two processes involved in randomizing patients to different interventions. First is choosing a randomization procedure to generate an unpredictable sequence of allocations; this may be a simple random assignment of patients to any of the groups at equal probabilities, may be "restricted," or may be "adaptive." A second and more practical issue is allocation concealment, which refers to the stringent precautions taken to ensure that the group assignment of patients are not revealed prior to definitively allocating them to their respective groups. Non-random "systematic" methods of group assignment, such as alternating subjects between one group and the other, can cause "limitless contamination possibilities" and can cause a breach of allocation concealment.[22]

Randomization procedures

An ideal randomization procedure would achieve the following goals[23]:

However, no single randomization procedure meets those goals in every circumstance, so researchers must select a procedure for a given study based on its advantages and disadvantages.

Simple randomization

This is a commonly used and intuitive procedure, similar to "repeated fair coin-tossing."[22] Also known as "complete" or "unrestricted" randomization, it is robust against both selection and accidental biases. However, its main drawback is the possibility of imbalanced group sizes in small RCTs. It is therefore recommended only for RCTs with over 200 subjects.[25]

Restricted randomization

To balance group sizes in smaller RCTs, some form of "restricted" randomization is recommended.[25] The major types of restricted randomization used in RCTs are:

Adaptive

At least two types of "adaptive" randomization procedures have been used in RCTs, but much less frequently than simple or restricted randomization:

Allocation concealment

"Allocation concealment" (defined as "the procedure for protecting the randomisation process so that the treatment to be allocated is not known before the patient is entered into the study") is considered desirable in RCTs.[27] In practice, in taking care of individual patients, clinical investigators in RCTs often find it difficult to maintain impartiality. Stories abound of investigators holding up sealed envelopes to lights or ransacking offices to determine group assignments in order to dictate the assignment of their next patient.[22] Such practices introduce selection bias and confounders (both of which should have minimized by randomization), thereby possibly distorting the results of the study.[22]

Some standard methods of ensuring allocation concealment include sequentially-numbered, opaque, sealed envelopes (SNOSE); sequentially-numbered containers; pharmacy controlled randomization; and central randomization.[22] It is recommended that allocation concealment methods be included in an RCT's protocol, and that the allocation concealment methods should be reported in detail in a publication of an RCT's results; however, 2005 study determined that most RCTs have unclear allocation concealment in their protocols, in their publications, or both.[28] On the other hand, a 2008 study of 146 meta-analyses concluded that the results of RCTs with inadequate or unclear allocation concealment tended to be biased toward beneficial effects only if the RCTs' outcomes were subjective as opposed to objective.[29]

Blinding

An RCT may be Blinded, (also called "masked") by "procedures that prevent study participants, caregivers, or outcome assessors from knowing which intervention was received."[29] Unlike allocation concealment, blinding is sometimes inappropriate or impossible to perform in an RCT; for example, if an RCT involves a treatment in which active participation of the patient is necessary (e.g., physical therapy), participants cannot be blinded to the intervention.

Traditionally, blinded RCTs have been classified as "single-blind," "double-blind," or "triple-blind"; however, in 2001 and 2006 two studies showed that these terms have different meanings for different people.[30][31] The 2010 CONSORT Statement specifies that authors and editors should not use the terms "single-blind," "double-blind," and "triple-blind"; instead, reports of blinded RCT should discuss "If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how."[2]

RCTs without blinding are referred to as "unblinded"[32], "open"[33], or (if the intervention is a medication) "open-label"[34]. In 2008 a study concluded that the results of unblinded RCTs tended to be biased toward beneficial effects only if the RCTs' outcomes were subjective as opposed to objective[29]; for example, in an RCT of treatments for multiple sclerosis, unblinded neurologists (but not blinded neurologists) felt that the treatments were beneficial[35]. In pragmatic RCTs, although the participants and providers are often unblinded, it is "still desirable and often possible to blind the assessor or obtain an objective source of data for evaluation of outcomes."[20]

Analysis of data from RCTs

The types of statistical methods used in RCTs depend on the characteristics of the data and include:

Regardless of the statistical methods used, important considerations in the analysis of RCT data include:

Reporting of RCT results

The CONSORT 2010 Statement is "an evidence-based, minimum set of recommendations for reporting RCTs."[40] The CONSORT 2010 checklist contains 25 items (many with sub-items) focusing on "individually randomised, two group, parallel trials" which are the most common type of RCT.[1] For other RCT study designs, "CONSORT extensions" have been published.[1]

Advantages

RCTs are considered by most to be the most reliable form of scientific evidence in the hierarchy of evidence that influences healthcare policy and practice because RCTs reduce spurious causality and bias. Results of RCTs may be combined in systematic reviews which are increasingly being used in the conduct of evidence-based medicine. Some examples of scientific organizations' considering RCTs or systematic reviews of RCTs to be the highest-quality evidence available are:

Notable RCTs with unexpected results that contributed to changes in clinical practice include:

Disadvantages

Many papers discuss the disadvantages of RCTs.[52][53] Among the most frequently-cited drawbacks are:

Limitations of external validity

The extent to which RCTs' results are applicable outside the RCTs varies; that is, RCTs' external validity may be limited.[52][54] Factors that can affect RCTs' external validity include[54]:

Costs

RCTs can be expensive[53]; one study found 28 Phase III RCTs funded by the National Institute of Neurological Disorders and Stroke prior to 2000 with a total cost of US$335 million[56], for a mean cost of US$12 million per RCT. Nevertheless, the return on investment of RCTs may be high, in that the same study projected that the 28 RCTs produced a "net benefit to society at 10-years" of 46 times the cost of the trials program, based on evaluating a quality-adjusted life year as equal to the prevailing mean per capita gross domestic product.[56]

Relative importance of RCTs and observational studies

Two studies published in The New England Journal of Medicine in 2000 found that observational studies and RCTs overall produced similar results[57][58]. The authors of the 2000 findings cast doubt on the ideas that "observational studies should not be used for defining evidence-based medical care" and that RCTs' results are "evidence of the highest grade."[57][58] However, a 2001 study published in Journal of the American Medical Association concluded that "discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common" between observational studies and RCTs.[59]

Two other lines of reasoning question RCTs' contribution to scientific knowledge beyond other types of studies:

Difficulty in studying rare events

Interventions to prevent events that occur only infrequently (e.g., sudden infant death syndrome) and uncommon adverse outcomes (e.g., a rare side effect of a drug) would require RCTs with extremely large sample sizes and may therefore best be assessed by observational studies.[52]

Difficulty in studying outcomes in distant future

It is costly to maintain RCTs for the years or decades that would be ideal for evaluating some interventions.[52][53]

Pro-industry findings in industry-funded RCTs

Some RCTs are fully or partly funded by the health care industry (e.g., the pharmaceutical industry) as opposed to government, nonprofit, or other sources. A systematic review published in 2003 found four 1986-2002 articles comparing industry-sponsored and nonindustry-sponsored RCTs, and in all the articles there was a correlation of industry sponsorship and positive study outcome.[63] A 2004 study of 1999-2001 RCTs published in leading medical and surgical journals determined that industry-funded RCTs "are more likely to be associated with statistically significant pro-industry findings."[64] One possible reason for the pro-industry results in industry-funded published RCTs is publication bias.[64]

Therapeutic misconception

Although subjects almost always provide informed consent for their participation in an RCT, studies since 1982 have documented that many RCT subjects believe that they are certain to receive treatment that is best for them personally; that is, they do not understand the difference between research and treatment.[65][66] Further research is necessary to determine the prevalence of and ways to address this "therapeutic misconception".[66]

Statistical error

RCTs are subject to both type I ("false positive") and type II ("false negative") statistical errors. Regarding Type I errors, a typical RCT will use 0.05 (i.e., 1 in 20) as the probability that the RCT will falsely find two equally effective treatments significantly different.[67] Regarding Type II errors, despite the publication of a 1978 paper noting that the sample sizes of many "negative" RCTs were too small to make definitive conclusions about the negative results[68], by 2005-2006 a sizeable proportion of RCTs still had inaccurate or incompletely-reported sample size calculations[69].

Cultural effects

The RCT method creates cultural effects that have not been well understood. [70] For example, patients with terminal illness may attempt to join trials as a last ditch attempt at treatment, even when treatments are unlikely to be successful.

RCTs in criminology, education, and international development

Criminology

A 2005 review found 83 randomized experiments in criminology published in 1982-2004, compared with only 35 published in 1957-1981.[71] The authors classified the studies they found into five categories: "policing", "prevention", "corrections", "court", and "community".[71] Focusing only on offending behavior programs, Hollin (2008) argued that RCTs may be difficult to implement (e.g., if an RCT required "passing sentences that would randomly assign offenders to programmes") and therefore that experiments with quasi-experimental design are still necessary.[72]

Education

RCTs have been used in evaluating a number of educational interventions. For example, a 2009 study randomized 260 elementary school teachers' classrooms to receive or not receive a program of behavioral screening, classroom intervention, and parent training, and then measured the behavioral and academic performance of their students.[73] Another 2009 study randomized classrooms for 678 first-grade children to receive a classroom-centered intervention, a parent-centered intervention, or no intervention, and then followed their academic outcomes through age 19.[74]

International development

RCTs are currently being used by a number of international development experts to measure the impact of development interventions worldwide. Development economists at research organizations including Abdul Latif Jameel Poverty Action Lab[75][76] and Innovations for Poverty Action[77] have used RCTs to measure the effectiveness of poverty, health, and education programs in the developing world. RCTs can be highly effective in policy evaluation since they allow researchers to isolate the impacts of a specific program from other factors such as other programs offered in the region, general macroeconomic growth, short-term events (such as a favorable harvest), and differences in personal qualities that might make one individual more successful than another.

For development economists, the main benefit to using RCTs compared to other research methods is that randomization guards against selection bias, a problem present in many current studies of development policy. In one notable example of a cluster RCT in the field of development economics, Olken (2007) randomized 608 villages in Indonesia in which roads were about to be built into six groups (no audit vs. audit, and no invitations to accountability meetings vs. invitations to accountability meetings vs. invitations to accountability meetings along with anonymous comment forms).[78] After estimating "missing expenditures" (a measure of corruption), Olken concluded that government audits were more effective than "increasing grassroots participation in monitoring" in reducing corruption.[78]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 Schulz KF, Altman DG, Moher D; for the CONSORT Group (2010). "CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials". Br Med J 340: c332. doi:10.1136/bmj.c332. PMID 20332509. 
  2. ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG (2010). "CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials". Br Med J 340: c869. doi:10.1136/bmj.c869. PMID 20332511. 
  3. ↑ Ranjith G (2005). "Interferon-α-induced depression: when a randomized trial is not a randomized controlled trial". Psychother Psychosom 74 (6): 387. doi:10.1159/000087787. PMID 16244516. 
  4. ↑ Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A (1981). "A method for assessing the quality of a randomized control trial". Control Clin Trials 2 (1): 31–49. doi:10.1016/0197-2456(81)90056-8. PMID 7261638. 
  5. ↑ Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG (1976). "Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design". Br J Cancer 34 (6): 585–612. PMID 795448. 
  6. ↑ Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG (1977). "Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples". Br J Cancer 35 (1): 1–39. PMID 831755. 
  7. ↑ Wollert KC, Meyer GP, Lotz J, Ringes-Lichtenberg S, Lippolt P, Breidenbach C, Fichtner S, Korte T, Hornig B, Messinger D, Arseniev L, Hertenstein B, Ganser A, Drexler H (2004). "Intracoronary autologous bone-marrow cell transfer after myocardial infarction: the BOOST randomised controlled clinical trial". Lancet 364 (9429): 141–8. doi:10.1016/S0140-6736(04)16626-9. PMID 15246726. 
  8. ↑ Streptomycin in Tuberculosis Trials Committee (1948). "Streptomycin treatment of pulmonary tuberculosis. A Medical Research Council investigation". Br Med J 2 (4582): 769–82. doi:10.1136/bmj.2.4582.769. PMID 18890300. 
  9. ↑ Brown D (1998-11-02). "Landmark study made research resistant to bias". Washington Post. 
  10. ↑ Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H (2006). "Comparison of effects in randomized controlled trials with observational studies in digestive surgery". Ann Surg 244 (5): 668–76. doi:10.1097/01.sla.0000225356.04304.bc. PMID 17060757. 
  11. ↑ 11.0 11.1 Stolberg HO, Norman G, Trop I (2004). "Randomized controlled trials". Am J Roentgenol 183 (6): 1539–44. PMID 15547188. http://www.ajronline.org/cgi/content/full/183/6/1539. 
  12. ↑ http://www.informaworld.com/smpp/content~db=all~content=a780179536~frm=abslink
  13. ↑ Meldrum ML (2000). "A brief history of the randomized controlled trial. From oranges and lemons to the gold standard". Hematol Oncol Clin North Am 14 (4): 745–60, vii. doi:10.1016/S0889-8588(05)70309-9. PMID 10949771. 
  14. ↑ Freedman B (1987). "Equipoise and the ethics of clinical research". N Engl J Med 317 (3): 141–5. doi:10.1056/NEJM198707163170304. PMID 3600702. 
  15. ↑ Gifford F (1995). "Community-equipoise and the ethics of randomized clinical trials". Bioethics 9 (2): 127–48. doi:10.1111/j.1467-8519.1995.tb00306.x. PMID 11653056. 
  16. ↑ Edwards SJL, Lilford RJ, Hewison J (1998). "The ethics of randomised controlled trials from the perspectives of patients, the public, and healthcare professionals". Br Med J 317 (7167): 1209–12. PMID 9794861. PMC 1114158. http://www.bmj.com/cgi/content/full/317/7167/1209. 
  17. ↑ Zelen M (1979). "A new design for randomized clinical trials". N Engl J Med 300 (22): 1242–5. doi:10.1056/NEJM197905313002203. PMID 431682. http://content.nejm.org/cgi/content/abstract/300/22/1242. 
  18. ↑ Torgerson DJ, Roland M (1998). "What is Zelen's design?". Br Med J 316 (7131): 606. PMID 9518917. PMC 1112637. http://www.bmj.com/cgi/content/full/316/7131/606. 
  19. ↑ 19.0 19.1 Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG (2010). "The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed". BMJ 340: c723. doi:10.1136/bmj.c723. PMID 20332510. PMC 2844941. http://www.bmj.com/cgi/content/full/340/mar23_1/c723. 
  20. ↑ 20.0 20.1 20.2 20.3 Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group (2008). "Improving the reporting of pragmatic trials: an extension of the CONSORT statement". BMJ 337: a2390. doi:10.1136/bmj.a2390. PMID 19001484. http://www.bmj.com/cgi/content/full/337/nov11_2/a2390. 
  21. ↑ 21.0 21.1 21.2 21.3 Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ; CONSORT Group (2006). "Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement". JAMA 295 (10): 1152–60. doi:10.1001/jama.295.10.1152. PMID 16522836. http://jama.ama-assn.org/cgi/content/full/295/10/1152. 
  22. ↑ 22.00 22.01 22.02 22.03 22.04 22.05 22.06 22.07 22.08 22.09 22.10 Schulz KF, Grimes DA (2002). "Generation of allocation sequences in randomised trials: chance, not choice". Lancet 359 (9305): 515–9. doi:10.1016/S0140-6736(02)07683-3. PMID 11853818. http://www.who.int/entity/rhl/LANCET_515-519.pdf. 
  23. ↑ 23.0 23.1 Lachin JM (1988). "Statistical properties of randomization in clinical trials". Control Clin Trials 9 (4): 289–311. doi:10.1016/0197-2456(88)90045-1. PMID 3060315. 
  24. ↑ Buyse ME (1989). "Analysis of clinical trial outcomes: some comments on subgroup analyses". Control Clin Trials 10 (4 Suppl): 187S–194S. doi:10.1016/0197-2456(89)90057-3. PMID 2605967. 
  25. ↑ 25.0 25.1 25.2 25.3 25.4 25.5 Lachin JM, Matts JP, Wei LJ (1988). "Randomization in clinical trials: conclusions and recommendations". Control Clin Trials 9 (4): 365–74. doi:10.1016/0197-2456(88)90049-9. PMID 3203526. 
  26. ↑ Rosenberger WF, Lachin JM (1993). "The use of response-adaptive designs in clinical trials". Control Clin Trials 14 (6): 471–84. doi:10.1016/0197-2456(93)90028-C. PMID 8119063. 
  27. ↑ Forder PM, Gebski VJ, Keech AC (2005). "Allocation concealment and blinding: when ignorance is bliss". Med J Aust 182 (2): 87–9. PMID 15651970. http://www.mja.com.au/public/issues/182_02_170105/for10877_fm.html. 
  28. ↑ Pildal J, Chan AW, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC (2005). "Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study". BMJ 330 (7499): 1049. doi:10.1136/bmj.38414.422650.8F. PMID 15817527. PMC 557221. http://www.bmj.com/cgi/content/full/330/7499/1049. 
  29. ↑ 29.0 29.1 29.2 Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008). "Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study". BMJ 336 (7644): 601–5. doi:10.1136/bmj.39465.451748.AD. PMID 18316340. PMC 2267990. http://www.bmj.com/cgi/content/full/336/7644/601. 
  30. ↑ Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Montori VM, Bhandari M, Guyatt GH (2001). "Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials". J Am Med Assoc 285 (15): 2000–3. doi:10.1001/jama.285.15.2000. PMID 11308438. http://jama.ama-assn.org/cgi/content/full/285/15/2000. 
  31. ↑ Haahr MT, Hróbjartsson A (2006). "Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors". Clin Trials 3 (4): 360–5. doi:10.1177/1740774506069153. PMID 17060210. http://ctj.sagepub.com/cgi/content/abstract/3/4/360. 
  32. ↑ Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, et al (2007). "The SANAD study of effectiveness of valproate, lamotrigine, or topiramate for generalised and unclassifiable epilepsy: an unblinded randomised controlled trial". Lancet 369 (9566): 1016–26. doi:10.1016/S0140-6736(07)60461-9. PMID 17382828. 
  33. ↑ Chan R, Hemeryck L, O'Regan M, Clancy L, Feely J (1995). "Oral versus intravenous antibiotics for community acquired lower respiratory tract infection in a general hospital: open, randomised controlled trial". BMJ 310 (6991): 1360–2. PMID 7787537. PMC 2549744. http://www.bmj.com/cgi/content/full/310/6991/1360. 
  34. ↑ Fukase K, Kato M, Kikuchi S, Inoue K, Uemura N, Okamoto S, Terao S, Amagai K, Hayashi S, Asaka M; Japan Gast Study Group (2008). "Effect of eradication of Helicobacter pylori on incidence of metachronous gastric carcinoma after endoscopic resection of early gastric cancer: an open-label, randomised controlled trial". Lancet 372 (9636): 392–7. doi:10.1016/S0140-6736(08)61159-9. PMID 18675689. 
  35. ↑ Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R (1994). "The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial". Neurology 44 (1): 16–20. PMID 8290055. http://www.neurology.org/cgi/content/abstract/44/1/16. 
  36. ↑ Manns MP, McHutchison JG, Gordon SC, Rustgi VK, Shiffman M, Reindollar R, Goodman ZD, Koury K, Ling M, Albrecht JK (2001). "Peginterferon alfa-2b plus ribavirin compared with interferon alfa-2b plus ribavirin for initial treatment of chronic hepatitis C: a randomised trial". Lancet 358 (9286): 958–65. doi:10.1016/S0140-6736(01)06102-5. PMID 11583749. 
  37. ↑ Schwartz GG, Olsson AG, Ezekowitz MD, Ganz P, Oliver MF, Waters D, Zeiher A, Chaitman BR, Leslie S, Stern T; Myocardial Ischemia Reduction with Aggressive Cholesterol Lowering (MIRACL) Study Investigators (2001). "Effects of atorvastatin on early recurrent ischemic events in acute coronary syndromes: the MIRACL study: a randomized controlled trial". J Am Med Assoc 285 (13): 1711–8. doi:10.1001/jama.285.13.1711. PMID 11277825. http://jama.ama-assn.org/cgi/content/full/285/13/1711. 
  38. ↑ 38.0 38.1 Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnson KC, Kotchen JM, Ockene J; Writing Group for the Women's Health Initiative Investigators (2002). "Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial". J Am Med Assoc 288 (3): 321–33. doi:10.1001/jama.288.3.321. PMID 12117397. http://jama.ama-assn.org/cgi/content/full/288/3/321. 
  39. ↑ Hollis S, Campbell F (1999). "What is meant by intention to treat analysis? Survey of published randomised controlled trials". Br Med J 319 (7211): 670–4. PMID 10480822. PMC 28218. http://www.bmj.com/cgi/content/full/319/7211/670. 
  40. ↑ CONSORT Group. "Welcome to the CONSORT statement Website". http://www.consort-statement.org. Retrieved 2010-03-29. 
  41. ↑ National Health and Medical Research Council (1998-11-16). A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: Commonwealth of Australia. p. 56. ISBN 1864960485. http://www.nhmrc.gov.au/_files_nhmrc/file/publications/synopses/cp30.pdf. Retrieved 2010-03-28. 
  42. ↑ 42.0 42.1 Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D; Methods Work Group, Third US Preventive Services Task Force (2001). "Current methods of the US Preventive Services Task Force: a review of the process". Am J Prev Med 20 (3 Suppl): 21–35. doi:10.1016/S0749-3797(01)00261-6. PMID 11306229. http://www.ahrq.gov/clinic/ajpmsuppl/review.pdf. 
  43. ↑ Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group (2008). "What is "quality of evidence" and why is it important to clinicians?". BMJ 336 (7651): 995–8. doi:10.1136/bmj.39490.551019.BE. PMID 18456631. PMC 2364804. http://www.bmj.com/cgi/content/full/336/7651/995. 
  44. ↑ Oxford Centre for Evidence-based Medicine (2009 March). "Levels of evidence". http://www.cebm.net/index.aspx?o=1025. Retrieved 2010-03-28. 
  45. ↑ 45.0 45.1 45.2 Anderson JL, Pratt CM, Waldo AL, Karagounis LA (1997). "Impact of the Food and Drug Administration approval of flecainide and encainide on coronary artery disease mortality: putting "Deadly Medicine" to the test". Am J Cardiol 79 (1): 43–7. doi:10.1016/S0002-9149(96)00673-X. PMID 9024734. http://www.ajconline.org/article/S0002-9149%2896%2900673-X/abstract. 
  46. ↑ 46.0 46.1 Rubin R (2006-10-16). "In medicine, evidence can be confusing - deluged with studies, doctors try to sort out what works, what doesn't". USA Today. http://www.usatoday.com/news/health/2006-10-15-medical-evidence-cover_x.htm. Retrieved 2010-03-22. 
  47. ↑ "Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. The Cardiac Arrhythmia Suppression Trial (CAST) Investigators". N Engl J Med 321 (6): 406–12. 1989. doi:10.1056/NEJM198908103210629. PMID 2473403. http://content.nejm.org/cgi/content/abstract/321/6/406. 
  48. ↑ Anderson GL, Limacher M, Assaf AR, Bassford T, Beresford SA, Black H, et al (2004). "Effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women's Health Initiative randomized controlled trial". JAMA 291 (14): 1701–12. doi:10.1001/jama.291.14.1701. PMID 15082697. http://jama.ama-assn.org/cgi/content/full/291/14/1701. 
  49. ↑ Grodstein F, Clarkson TB, Manson JE (2003). "Understanding the divergent data on postmenopausal hormone therapy". N Engl J Med 348 (7): 645–50. doi:10.1056/NEJMsb022365. PMID 12584376. http://content.nejm.org/cgi/content/extract/348/7/645. 
  50. ↑ Vandenbroucke JP (2009). "The HRT controversy: observational studies and RCTs fall in line". Lancet 373 (9671): 1233–5. doi:10.1016/S0140-6736(09)60708-X. PMID 19362661. http://linkinghub.elsevier.com/retrieve/pii/S0140-6736(09)60708-X. 
  51. ↑ Hsu A, Card A, Lin SX, Mota S, Carrasquillo O, Moran A (2009). "Changes in postmenopausal hormone replacement therapy use among women with high cardiovascular risk". Am J Public Health 99 (12): 2184–7. doi:10.2105/AJPH.2009.159889. PMID 19833984. http://ajph.aphapublications.org/cgi/content/full/99/12/2184. 
  52. ↑ 52.0 52.1 52.2 52.3 52.4 Black N (1996). "Why we need observational studies to evaluate the effectiveness of health care". BMJ 312 (7040): 1215–8. PMID 8634569. PMC 2350940. http://www.bmj.com/cgi/content/full/bmj%3B312/7040/1215. 
  53. ↑ 53.0 53.1 53.2 Sanson-Fisher RW, Bonevski B, Green LW, D'Este C (2007). "Limitations of the randomized controlled trial in evaluating population-based health interventions". Am J Prev Med 33 (2): 155–61. doi:10.1016/j.amepre.2007.04.007. PMID 17673104. http://www.ajpm-online.net/article/S0749-3797%2807%2900225-5/abstract. 
  54. ↑ 54.0 54.1 Rothwell PM (2005). "External validity of randomised controlled trials: "to whom do the results of this trial apply?"". Lancet 365 (9453): 82–93. doi:10.1016/S0140-6736(04)17670-8. PMID 15639683. http://apps.who.int/rhl/Lancet_365-9453.pdf. 
  55. ↑ Van Spall HG, Toren A, Kiss A, Fowler RA (2007). "Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review". JAMA 297 (11): 1233–40. doi:10.1001/jama.297.11.1233. PMID 17374817. http://jama.ama-assn.org/cgi/content/full/297/11/1233. 
  56. ↑ 56.0 56.1 Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS (2006). "Effect of a US National Institutes of Health programme of clinical trials on public health and costs". Lancet 367 (9519): 1319–27. doi:10.1016/S0140-6736(06)68578-4. PMID 16631910. http://www.chrp.org/pdf/HSR20070511.pdf. 
  57. ↑ 57.0 57.1 Benson K, Hartz AJ (2000). "A comparison of observational studies and randomized, controlled trials". N Engl J Med 342 (25): 1878–86. doi:10.1056/NEJM200006223422506. PMID 10861324. http://content.nejm.org/cgi/content/full/342/25/1878. 
  58. ↑ 58.0 58.1 Concato J, Shah N, Horwitz RI (2000). "Randomized, controlled trials, observational studies, and the hierarchy of research designs". N Engl J Med 342 (25): 1887–92. doi:10.1056/NEJM200006223422507. PMID 10861325. PMC 1557642. http://nejm.highwire.org/cgi/content/full/342/25/1887. 
  59. ↑ Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J (2001). "Comparison of evidence of treatment effects in randomized and nonrandomized studies". J Am Med Assoc 286 (7): 821–30. doi:10.1001/jama.286.7.821. PMID 11497536. http://jama.ama-assn.org/cgi/content/full/286/7/821. 
  60. ↑ Vandenbroucke JP (2008). "Observational research, randomised trials, and two views of medical science". PLoS Med 5 (3): e67. doi:10.1371/journal.pmed.0050067. PMID 18336067. 
  61. ↑ 61.0 61.1 Glasziou P, Chalmers I, Rawlins M, McCulloch P (2007). "When are randomised trials unnecessary? Picking signal from noise". Br Med J 334 (7589): 349–51. doi:10.1136/bmj.39070.527986.68. PMID 17303884. 
  62. ↑ Einhorn LH (2002). "Curing metastatic testicular cancer". Proc Natl Acad Sci U S A 99 (7): 4592–5. doi:10.1073/pnas.072067999. PMID 11904381. 
  63. ↑ Bekelman JE, Li Y, Gross CP (2003). "Scope and impact of financial conflicts of interest in biomedical research: a systematic review". J Am Med Assoc 289 (4): 454–65. doi:10.1001/jama.289.4.454. PMID 12533125. http://jama.ama-assn.org/cgi/content/full/289/4/454. 
  64. ↑ 64.0 64.1 Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ (2004). "Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials". Can Med Assoc J 170 (4): 477–80. PMID 14970094. PMC 332713. http://ecmaj.com/cgi/content/full/170/4/477. 
  65. ↑ Appelbaum PS, Roth LH, Lidz C (1982). "The therapeutic misconception: informed consent in psychiatric research". Int J Law Psychiatry 5 (3-4): 319–29. doi:10.1016/0160-2527(82)90026-7. PMID 6135666. 
  66. ↑ 66.0 66.1 Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, Joffe S, Kass N, King NM, Lidz CW, Miller FG, Nelson DK, Peppercorn J, Rothschild BB, Sankar P, Wilfond BS, Zimmer CR (2007). "Clinical trials and medical care: defining the therapeutic misconception". PLoS Med 4 (11): e324. doi:10.1371/journal.pmed.0040324. PMID 18044980. 
  67. ↑ Wittes J (2002). "Sample size calculations for randomized controlled trials". Epidemiol Rev 24 (1): 39–53. doi:10.1093/epirev/24.1.39. PMID 12119854. 
  68. ↑ Freiman JA, Chalmers TC, Smith H Jr, Kuebler RR (1978). "The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials". N Engl J Med 299 (13): 690–4. doi:10.1056/NEJM197809282991304. PMID 355881. http://content.nejm.org/cgi/content/abstract/299/13/690. 
  69. ↑ Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P (2009 May 12). "Reporting of sample size calculation in randomised controlled trials: review". Br Med J 338: b1732. doi:10.1136/bmj.b1732. PMID 19435763. 
  70. ↑ Jain SL (2010). "The mortality effect: counting the dead in the cancer trial". Public Culture 21 (1): 89–117. doi:10.1215/08992363-2009-017. 
  71. ↑ 71.0 71.1 Farrington DP, Welsh BC (2005). "Randomized experiments in criminology: What have we learned in the last two decades?". Journal of Experimental Criminology 1 (1): 9–38. doi:10.1007/s11292-004-6460-0. 
  72. ↑ Hollin CR (2008). "Evaluating offending behaviour programmes: does only randomization glister?". Criminology and Criminal Justice 8 (1): 89–106. doi:10.1177/1748895807085871. 
  73. ↑ Walker HM, Seeley JR, Small J, Severson HH, Graham BA, Feil EG, Serna L, Golly AM, Forness SR (2009). "A randomized controlled trial of the First Step to Success early intervention. Demonstration of program efficacy outcomes in a diverse, urban school district". Journal of Emotional and Behavioral Disorders 17 (4): 197–212. doi:10.1177/1063426609341645. 
  74. ↑ Bradshaw CP, Zmuda JH, Kellam SG, Ialongo NS (2009). "Longitudinal impact of two universal preventive interventions in first grade on educational outcomes in high school". Journal of Educational Psychology 101 (4): 926–937. doi:10.1037/a0016586. 
  75. ↑ Duflo E, Glennerster R, Kremer M (2006 December 12). "Using randomization in development economics research: a toolkit" (PDF). MIT Department of Economics. http://web.archive.org/web/20070119193234/http://www.povertyactionlab.com/papers/Using+Randomization+in+Development+Economics.pdf.  Working Paper No. 06-36
  76. ↑ Banerjee AV, Cole S, Duflo E, Linden L (2007). "Remedying education: evidence from two randomized experiments in India". Quarterly Journal of Economics 122 (3): 1235–1264. doi:10.1162/qjec.122.3.1235. 
  77. ↑ Karlan D, Zinman J (2010). "Expanding credit access: using randomized supply decisions to estimate the impacts". Review of Financial Studies 23 (1): 433–464. doi:10.1093/rfs/hhp092. 
  78. ↑ 78.0 78.1 Olken BA (2007). "Monitoring corruption: evidence from a field experiment in Indonesia". Journal of Political Economy 115 (2): 200–249. doi:10.1086/517935. 

Further reading

External links

Biomedical research: Clinical study design / Design of experiments
Overview

Clinical trial · Clinical trial protocol · Clinical trial management · Academic clinical trials
Controlled study
(EBM I to II-1; A to B)

Randomized controlled trial (Blind experiment, Open-label trial)
Observational study
(EBM II-2 to II-3; B to C)

Cross-sectional study vs. Longitudinal study, Ecological study
Cohort study (Retrospective cohort study, Prospective cohort study)
Case-control study (Nested case-control study)
Case series · Case study / Case report
Epidemiology/
methods

occurrence: Incidence (Cumulative incidence) · Prevalence (Point prevalence, Period prevalence)
association: absolute (Absolute risk reduction, Attributable risk, Attributable risk percent) · relative (Relative risk, Odds ratio, Hazard ratio)
other: Virulence · Infectivity · Mortality rate · Morbidity · Case fatality · Specificity and sensitivity · Likelihood-ratios
Trial/test types

In vitro / In vivo · Animal testing · Animal testing on non-human primates · First-in-man study · Multicenter trial · Seeding trial · Vaccine trial
Analysis of clinical trials

Risk-benefit analysis
Interpretation of results

Selection bias · Correlation does not imply causation · Null result
Category · Glossary · List of topics
Design of experiments
Scientific
Method
Scientific experiment · Statistical design · Control · Internal & external validity · Experimental unit · Blinding
Optimal design: Bayesian  · Random assignment · Randomization · Restricted randomization · Replication versus subsampling · Sample size
Treatment
& Blocking
Treatment · Effect size · Contrast · Interaction · Confounding · Orthogonality · Blocking · Covariate · Nuisance variable
Models
& Inference
Linear regression  · Ordinary least squares  · Bayesian
Random effect · Mixed model · Hierarchical model: Bayesian
Analysis of variance (Anova) · Cochran's theorem · Manova (multivariate)  · Ancova (covariance)
Compare means · Multiple comparison ·
Designs:

Completely
Randomized

Factorial · Fractional factorial · Plackett-Burman  · Taguchi
Response-surface design · Polynomial & rational modeling · Box-Behnken · Central composite

Block  · Generalized randomized block design (GRBD)  · Latin square  · Graeco-Latin square · Hyper-Graeco-Latin square · Latin hypercube
Repeated measures experiment · Crossover experiment ·
Randomized controlled trial · Sequential analysis · Sequential probability ratio test
Glossary · Category · Statistics portal · Statistical outline · Statistical topics
Statistics
Category · Portal · Outline · Index